Effects  of  Crew-Aiding  Behaviors  on  Soldier  Performance 
During  Target  Engagement  Tasks  in  a  Virtual 
Battlefield  Simulation 


by  Chuck  H.  Perala,  Bruce  S.  Sterling,  Steve  Scheiner,  and  Deborah  Butler 


ARL-TR-4026 


February  2007 


Approved  for  public  release;  distribution  is  unlimited. 


NOTICES 

Disclaimers 

The  findings  in  this  report  are  not  to  be  eonstrued  as  an  offieial  Department  of  the  Army  position 
unless  so  designated  by  other  authorized  doeuments. 

Citation  of  manufaeturer’s  or  trade  names  does  not  eonstitute  an  offieial  endorsement  or 
approval  of  the  use  thereof 


DESTRUCTION  NOTICE — ^Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return  it  to 
the  originator. 


Army  Research  Laboratory 

Aberdeen  Proving  Ground,  MD  21005-5425 


ARL-TR-4026 _ February  2007 


Effects  of  Crew-Aiding  Behaviors  on  Soldier  Performance 
During  Target  Engagement  Tasks  in  a  Virtual 
Battlefield  Simulation 

Chuck  H.  Perala  and  Bruce  S.  Sterling 
Human  Research  &  Engineering  Directorate,  ARE 

Steve  Scheiner 
DCS  Corporation 

Deborah  Butler 

U.S.  Army  Aviation  and  Missile  Research,  Development,  &  Engineering  Center 


Approved  for  public  release;  distribution  is  unlimited. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMBNo.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and 
Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no 
person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  0MB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

This  research  examined  the  impact  of  crew-aiding  behaviors  (CABs)  on  Soldier  workload,  stress,  situation  awareness,  and 
performance  in  a  laboratory  setting.  Specifically,  this  experiment  examined  the  effectiveness  of  CABs  designed  to  prioritize 
targets  (based  on  threat  level  and  proximity)  and  provide  weapons  platform  and  munition  recommendations  to  service  each 
target.  This  condition  was  compared  with  a  NoCAB  or  manual  condition  in  which  participants  performed  the  same  task  of 
prioritizing  and  engaging  targets  without  the  use  of  the  CABs. 

Results  showed  that  CABs  significantly  reduced  time  and  workload  when  participants  conducted  the  task  of  prioritizing  and 
engaging  targets.  Participants  took  significantly  less  time  to  complete  the  prioritization  and  engagement  task  when  using  CABs 
versus  when  they  performed  the  same  task  manually  (i.e.,  the  NoCAB  condition).  Overall  task  time  was  reduced  by  36%  when 
CABs  were  used.  Overall  workload,  as  well  as  the  subscales  of  mental,  temporal,  and  effort  workload,  were  significantly 
reduced  when  CABs  were  used.  Overall  workload  was  28%  less  when  CABs  were  used  versus  when  they  were  not.  Mental 
and  temporal  workload  were  both  46%  less  when  CABs  were  used  versus  when  they  were  not,  and  effort  workload  was  36% 
less  when  were  used  versus  when  they  were  not. 


15.  SUBJECT  TERMS 

CAB;  crew-aiding  behavior;  decision  aids;  target  prioritization 

17.  LIMITATION  18.  NUMBER 
OF  ABSTRACT  OF  PAGES 

SAR  50 

standard  Form  298  (Rev.  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


19a.  NAME  OF  RESPONSIBLE  PERSON 

Chuck  H.  Perala 

19b.  TELEPHONE  NUMBER  {Include  area  code) 

502-624-8778 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

UNCLASSIFIED 

UNCLASSIFIED 

UNCLASSIFIED 

1.  REPORT  DATE  (DD-MM-ryyy;  2.  REPORT  TYPE 

February  2007  Final 

4.  TITLE  AND  SUBTITLE 

Effects  of  Crew- Aiding  Behaviors  on  Soldier  Performance  During  Target 
Engagement  Tasks  in  a  Virtual  Battlefield  Simulation 

6.  AUTHOR(S) 

Chuck  H.  Perala  and  Bruce  S.  Sterling  (both  of  ARE);  Steve  Scheiner  (DCS 
Corp);  Deborah  Butler  (AMRDEC) 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Laboratory 

Human  Research  &  Engineering  Directorate 
Aberdeen  Proving  Ground,  MD  21005-5425 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


3.  DATES  COVERED  (From  -  To) 

June  through  July  2006 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 

7MB25R 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

ARL-TR-4026 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT  NUMBERS 


11 


Contents 


List  of  Figures  v 

List  of  Tables  v 

Acknowledgments  vi 

1.  Introduction  1 

1 . 1  Background . 1 

1.2  Research  Objective . 2 

2.  Method  2 

2.1  Apparatus . 2 

2.2  Performance  Measures . 3 

2.2.1  Workload . 3 

2.2.2  Stress . 3 

2.2.3  Situation  Awareness . 3 

2.2.4  Task  Time . 4 

2.3  Participants . 4 

2.4  Training . 4 

2.5  Procedures . 4 

2.5.1  Overview . 4 

2.5.2  Stepwise  Proeedures . 5 

2.6  Analysis . 6 

3.  Results  6 

3.1  Overall  Workload . 6 

3.2  Workload  Subseales . 8 

3.2.1  Mental  Workload . 11 

3.2.2  Temporal  Workload . 11 

3.2.3  Effort  Workload . 11 

3.3  Stress . 13 

3 .4  Situation  Awareness . 16 


iii 


3.5  Task  Time . 17 

3.5.1  Engagement-only  Analysis . 1 8 

4.  Discussion  21 

4.1  Workload . 21 

4.2  Stress . 22 

4.3  Task  Time . 22 

5.  Conclusion  23 

6.  References  24 

Appendix  A,  SIL  Crew  Station  Layout  and  Functionality  Descriptions  25 

Appendix  B.  Surveys  35 

Distribution  List  41 


IV 


List  of  Figures 


Figure  1 .  CAT  SIL  crew  station  simulator . 2 

Figure  2.  CAT  SIL  displays . 3 

Figure  3.  Comparison  of  overall  workload  across  conditions . 8 

Figure  4.  Comparison  of  mental  workload  across  conditions . 12 

Figure  5.  Comparison  of  temporal  workload  across  conditions . 12 

Figure  6.  Comparison  of  temporal  workload  across  conditions . 13 

Figure  7.  Workload  subscale  comparison  across  conditions  (performance  is  a  reverse  scale). ..  13 

Figure  8.  Comparison  of  physical  and  mental  stress  across  conditions . 15 

Figure  9.  Comparison  of  situation  awareness  across  conditions . 17 

Figure  10.  Average  time  to  complete  task . 19 

Figure  1 1 .  Condition  x  Presentation  Order  interaction . 20 

Figure  12.  Average  engagement-only  times . 20 


List  of  Tables 


Table  1.  Means  for  overall  workload . 7 

Table  2.  MANOVA  for  overall  workload . 7 

Table  3.  ANOVA  for  overall  workload . 7 

Table  4.  Tukey-Kramer  comparisons  for  overall  workload-dependent  variables . 8 

Table  5.  Means  for  workload  subscales . 9 

Table  6.  MANOVA  for  workload  subscales . 9 

Table  7.  ANOVA  for  workload  subscales . 10 

Table  8.  Tukey-Kramer  comparisons  for  workload  subscales . 10 

Table  9.  Means  for  mental  stress . 14 

Table  10.  MANOVA  for  mental  stress . 14 

Table  11.  Means  for  physical  stress . 15 

Table  12.  MANOVA  for  physical  stress . 15 

Table  13.  ANOVA  for  physical  stress . 15 

Table  14.  Means  for  situation  awareness . 16 

Table  15.  MANOVA  for  situation  awareness . 16 

Table  16.  Means  for  task  time . 18 

Table  17.  MANOVA  for  task  time . 18 

Table  18.  ANOVA  for  task  time . 19 

Table  19.  Means  for  task  time  and  presentation  order . 19 


V 


Acknowledgments 


This  study  was  a  joint  effort  among  the  U.S.  Army  Researeh  Laboratory;  the  U.S.  Army  Tank- 
Automotive  Command  Researeh,  Development,  and  Engineering  Center;  the  U.S.  Army  Aviation 
and  Missile  Research,  Development,  &  Engineering  Center;  the  Unit  of  Action  Maneuver  Battle 
Eaboratory  (UAMBE);  DCS  Corporation;  and  Alion  Science  and  Technology.  The  authors  would 
like  to  thank  the  many  representatives  from  these  organizations  whose  dedication  and  hard  work 
made  Research,  Development,  and  Engineering  Command  (RDECOM)-UAMBE  (RUX06)  a 
success. 


1.  Introduction 


As  the  U.S.  Army  network-centric  digital  battlefields  continue  to  expand,  so  do  the  workload 
demands  placed  on  Soldiers  who  use  the  increasing  amount  of  information  to  conduct  their 
missions.  In  an  effort  to  reduce  workload  and  stress  for  these  Soldiers,  decision  aids,  called  crew- 
aiding  behaviors  (CABs),  have  been  developed,  which  provide  a  level  of  automation  designed  to 
assist  Soldiers  in  the  performance  of  their  tasks.  A  field-based  experiment  was  conducted  to 
assess  the  effects  of  these  decision  aids  on  Soldier  performance  in  a  simulated  battlefield  environ¬ 
ment.  We  evaluated  the  effects  of  the  CABs  by  measuring  and  comparing  levels  of  task  time, 
workload,  stress,  and  situation  awareness  between  two  experimental  conditions.  The  experimen¬ 
tal  task  was  target  prioritization,  weapon  system  and  munition  matching,  and  target  engagement 
with  and  without  the  use  of  the  decision  aids.  This  report  describes  the  details  associated  with  the 
experiment,  statistical  methods  used,  analytical  results,  and  an  interpretation  and  discussion  of 
those  results. 

1.1  Background 

Soldiers  on  future  battlefields  will  have  a  greater  number  of  simultaneous  tasks  to  perform  while 
conducting  their  missions.  They  must  maintain  local  security  to  guard  against  threats  from  enemy 
vehicles  or  dismounted  threats  with  hand-held  weapons.  They  must  be  able  to  receive  instructions 
from  platoon  level  or  above  and  provide  relevant  information  to  their  leaders  and  subordinates. 
Soldiers  are  also  responsible  for  the  mobility  of  their  platform  and  plotting  routes  for  unmanned 
aerial  and  ground  platforms  under  their  control.  They  must  also  monitor  sensor  information  from 
these  unmanned  systems,  interpret  the  information,  and  provide  relevant  portions  to  their  chain  of 
command.  In  addition  to  this,  they  must  seek,  close  with,  and  destroy  enemy  forces  of  disparate 
types  and  strengths.  Decision  aids,  also  known  as  CABs,  may  prove  useful  in  assisting  Soldiers 
in  the  performance  of  these  tasks.  CABs  may  consist  of  software  designed  to  autonomously  drive 
vehicles,  plot  routes  for  road  marches,  plot  tactical  routes  to  observation  points  (OPs),  establish 
effective  support-by-fire  missions,  or  prioritize  targets  for  engagement.  These  CABs  should 
reduce  workload  and  stress  for  Soldiers,  as  well  as  improve  other  measures  of  performance  (such 
as  reduce  target  acquisition  and  engagement  time). 

This  experiment,  known  as  the  Lethality  Experiment,  was  one  of  several  experiments  conducted 
under  the  name  of  RDECOM-UAMBL  (RUX06).  These  experiments  were  conducted  jointly 
between  the  U.S.  Army  Research  Development  and  Engineering  Command  (RDECOM), 
specifically,  the  U.S.  Army  Research  Eaboratory’s  (ARE’s)  Human  Research  and  Engineering 
Directorate;  Tank  Automotive  Research  Development  and  Engineering  Center  (TARDEC); 
Aviation  and  Missile  Research  Development  and  Engineering  Center  (AMRDEC);  and  the  Unit 
of  Action  Maneuver  Battle  Eab  (UAMBE)  in  support  of  the  Crew-integration  and  Automation 
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Test  bed  (CAT)  Advaneed  Technology  Demonstration  (ATD)  program,  or  CAT-ATD.  Experi¬ 
mentation  was  conducted  at  Fort  Knox,  Kentucky,  in  July  2006. 

1,2  Research  Objective 

The  objective  of  this  research  was  to  determine  the  impact  of  CABs  on  Soldier  workload,  stress, 
situation  awareness,  and  performance.  Specifically,  this  experiment  examined  the  effectiveness 
of  CABs  designed  to  prioritize  targets  (based  on  threat  level  and  proximity)  and  provide  weapons 
platform  and  munition  recommendations  to  service  each  target. 


2.  Method 


2,1  Apparatus 

This  experiment  took  place  entirely  in  simulation;  however,  the  crew  station  was  identical  to  that 
used  in  the  actual  field  vehicle.  The  Soldier- in-the-loop  (SIL)  interface  (see  figure  1)  consisted 
of  three  vertically  oriented  liquid  crystal  displays  situated  in  an  arc  in  front  of  a  seated  participant. 
Each  display  was  divided  in  two,  horizontally,  with  information  on  each  of  the  six  “screens”  pro¬ 
vided  from  various  computer  systems,  which  were  transparent  to  the  SIE  operation  and  the  partici¬ 
pant.  Figure  2  shows  the  basic  layout  of  the  three  displays  (six  screens)  used  during  this  experi¬ 
ment,  with  the  target  prioritization  list  on  the  center  display.  Participants  could  select  targets  and 
weapons  by  touching  on-screen  buttons  or  by  scrolling  through  the  list  using  a  thumb  button  on  the 
driver’s  yoke.  The  yoke  was  also  used  to  slew  the  weapon  system  and  to  engage  each  target. 
Detailed  information  regarding  each  screen  and  button  functions  is  available  in  appendix  A. 


Figure  1.  CAT  SIL  crew  station  simulator. 
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Figure  2.  CAT  SIL  displays. 


2,2  Performance  Measures 

2.2.1  Workload 

To  measure  subjective  self-ratings  of  perceived  workload,  the  National  Aeronautics  and  Space 
Administration  (NASA)-Task  Load  Index  (TLX)  was  used.  The  NASA-TLX  (Hart  &  Staveland, 
1988)  is  a  multi-dimensional  rating  procedure  that  derives  an  overall  workload  score  based  on 
ratings  from  six  subscales.  The  subscales  include  Mental  Demands,  Physical  Demands,  Temporal 
Demands,  Own  Performance,  Effort,  and  Frustration.  Ratings  were  collected  via  questionnaire. 
This  instrument  is  included  in  appendix  B. 

2.2.2  Stress 

One-item  rating  scales  measuring  physical  stress  and  mental  stress  were  used.  These  scales  are 
shown  in  appendix  B. 

2.2.3  Situation  Awareness 

The  measure  of  situation  awareness  (SA)  used  was  a  variation  of  the  China  Lake  Situation 
Awareness  (CLSA)  rating  scale  developed  by  Adams  (1998).  The  original  CSLA  is  a  five-point 
scale  developed  by  the  Naval  Air  Warfare  Center  to  measure  SA  in  flight.  The  version  used  here 
is  a  10-point  scale  adapted  to  measure  SA  more  generally.  This  measure  is  shown  in  appendix  B. 
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2,2,4  Task  Time 

The  measure  of  task  time  eonsisted  of  the  total  time  required  to  eomplete  the  task  of  prioritizing 
and  servieing  targets.  These  measures  were  recorded  manually  by  the  experimenter  during  trial 
runs. 


2.3  Participants 

Twelve  active-duty  male  Soldiers  volunteered  for  this  experiment.  One  Soldier  was  a  Captain 
(03),  seven  Soldiers  were  Sergeants  First  Class  (E7),  and  four  Soldiers  were  Staff  Sergeants  (E6). 
Military  occupational  specialties  were  primarily  Ml  Armor  Crewmen  (19K).  Nine  participants 
were  19K,  one  19D  (Cavalry  Scout),  one  14E  (Patriot  Eire  Control  Enhanced  Operator),  and  one 
25B  (Information  Systems  Operator- Analyst). 

2.4  Training 

Participants  were  given  a  1-hour  block  of  instruction  and  practice  for  the  task  of  prioritizing  and 
servicing  a  list  of  targets,  in  both  CAB  and  NoCAB  conditions.  The  instruction  consisted  of 
familiarization  with  the  displays  and  controls  and  a  detailed  explanation  of  the  tasks,  conditions, 
and  standards  expected  during  experimentation.  Depending  on  which  condition  was  presented 
first,  training  for  that  condition  was  presented  before  experimentation.  Eor  example,  if  a  partici¬ 
pant  was  testing  in  the  CAB  condition  first,  the  CAB  training  was  conducted  before  testing  in  the 
CAB  condition.  Eollowing  testing  and  a  short  break,  the  NoCAB  training  was  conducted  before 
the  NoCAB  test. 

2.5  Procedures 
2,5,1  Overview 

Participants  were  randomly  assigned  a  participant  identification  number  and  then  given  an  over¬ 
view  of  the  experiment  and  familiarization  with  the  surveys.  Baseline  measures  of  workload, 
stress,  and  SA  were  taken  before  each  trial.  Eor  stress,  participants  were  asked  to  rate  how  they 
felt  “right  now”.  Eor  baseline  workload  and  SA,  participants  were  asked  to  rate  the  level  of 
workload  and  SA  they  experienced  while  driving  to  work  that  morning. 

Participants  were  then  trained  and  tested  in  target  prioritization,  weapons  and  munitions  selection, 
and  target  engagement  procedures;  again,  one  condition  was  taught  and  tested  first  and  then  the 
other.  Half  of  the  participants  engaged  targets  first  using  CABs,  and  half  engaged  targets  first 
without  using  CABs.  Each  participant  tested  in  both  CAB  and  NoCAB  conditions.  After  each 
experimental  trial,  participants  completed  the  workload,  stress,  and  SA  surveys. 

In  both  trials,  the  participant  was  notionally  task  organized  with  an  Armed  Robotic  Vehicle- 
Assault  (ARV-A)  and  Armed  Robotic  Vehicle-Reconnaissance  (ARV-R).  The  participant’s 
Mounted  Combat  System  (MCS)  platoon  was  providing  fire  support  for  a  Reconnaissance  and 
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Surveillance  (R&S)  Vehicle  Company.  The  participant’s  platoon  leader  detected  a  potential 
threat  at  approxi-mately  2.5  kilometers  from  the  participant’s  vehicle,  which  required  additional 
R&S.  The  platoon  leader  instructed  the  participant  to  use  the  ARV-R  to  investigate.  The 
participant’s  mission  began  after  the  ARV-R  reconnaissance  was  complete.  The  participant’s  task 
was  to  analyze  and  prioritize  all  targets  that  were  presented  in  the  target  queue,  select  the 
appropriate  weapons  platform  and  munition  type,  and  engage  and  defeat  as  many  of  those  threats 
as  possible,  as  quickly  as  possible.  All  targets  were  presented  simultaneously. 

2,5.2  Stepwise  Procedures 

Participants  were  seated  at  the  SIL  station  (see  figure  1)  and  given  a  1-hour  block  of  instruction 
regarding  each  facet  of  the  interface,  controls,  functions,  and  required  task,  conditions,  and 
standards.  After  this  block  of  instruction,  participants  took  a  short  break  and  then  were  asked  to 
practice  using  the  SIL  in  a  manner  consistent  with  the  experimental  tasks.  This  practice  session 
was  for  the  particular  trial  with  which  the  participant  started.  For  example,  if  he  started  with  Trial 
1  first,  this  practice  session  was  in  the  NoCAB  mode.  Conversely,  if  he  started  with  Trial  2  first, 
this  practice  session  was  in  the  CAB  mode.  Odd-numbered  participants  were  administered  Trial  1 
first  and  even-numbered  participants  were  administered  Trial  2  first.  Once  the  instructor  and 
participant  were  satisfied  with  the  participant’s  level  of  understanding,  comfort,  and  proficiency 
with  the  system,  the  experimental  trial  was  administered.  During  each  trial  run,  the  experimenter 
timed  each  target  prioritization  and  engagement  sequence  from  start  to  finish.  After  each  trial, 
participants  were  asked  to  complete  the  workload,  stress,  and  SA  surveys.  Following  a  short 
break,  participants  were  allowed  to  practice  before  the  second  and  final  trial. 

2.5.2. 1  Trial  1 

This  trial  entailed  prioritizing  a  list  of  targets  and  matching  weapons  to  threats  without  the  use  of 
CABs.  The  participant  was  presented  with  a  list  of  targets  in  the  target  queue.  This  list  was  not 
sorted  or  prioritized  in  any  way,  and  no  weapon  or  munition  recommendations  were  given.  The 
task  was  to  prioritize  the  list  based  on  proximity  and  threat  and  to  select  the  appropriate  weapon 
system  and  munition  for  engagement.  The  next  step  was  to  engage  and  destroy  the  target.  After 
destroying  the  target,  the  participant  deleted  that  target  from  the  target  queue  and  moved  to  the 
next  target.  The  trial  ended  when  all  targets  were  destroyed  or  all  munitions  from  each  weapons 
platform  were  expended.  Surveys  were  completed  at  the  end  of  the  trial. 

2.5. 2.2  Trial  2 

This  trial  entailed  servicing  a  list  of  targets  that  were  sorted  and  prioritized  automatically  by  the 
CABs.  The  task  was  to  verify  and  select  each  target  in  the  target  queue,  verify  the  recommended 
weapon  and  munition,  engage  and  destroy  the  target,  delete  the  target  from  the  target  queue  after 
verifying  its  destruction,  and  then  move  to  the  next  target  in  the  target  queue.  The  trial  ended 
when  all  targets  were  destroyed  or  all  munitions  from  each  weapons  platform  were  expended. 
Surveys  were  completed  at  the  end  of  the  trial. 
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2,6  Analysis 


All  dependent  measures  (workload,  stress,  and  SA)  were  analyzed  with  the  use  of  a  mixed  model 
analysis  of  varianee  (ANOVA)  with  one  between-subjeet  faetor  (order  of  eonditions)  with  two 
levels  (CAB  first  andNoCAB  first)  and  one  within-subjeet  faetor  (eondition)  with  three  levels 
(baseline,  CAB,  and  NoCAB).  Performanee  measures  were  analyzed  with  a  mixed  model 
ANOVA  with  one  between-subjeet  faetor  (order  of  eonditions)  with  two  levels  (CAB  first  and 
NoCAB  first)  and  one  within-subjeet  faetor  (eondition)  with  two  levels  (CAB  and  NoCAB). 


3.  Results 


A  multivariate  ANOVA  (MANOVA)  was  eondueted  separately  on  eaeh  of  the  data  sets  eolleeted 
from  the  experiment.  This  was  done  sinee  the  data  eolleeted  were  repeated  measures.  Sinee 
spherieity  assumptions  are  not  made  in  the  MANOVA,  this  test  is  eonsidered  to  be  “exaet”  for 
repeated  measures  designs,  while  the  univariate  approaeh  ean  only  be  eonsidered  “approximate” 
(Vasey  &  Thayer,  1987). 

Based  upon  the  MANOVA  results,  individual  ANOVAs  were  eondueted  on  eaeh  data  set.  Only 
those  significant  main  effects  and  interactions  found  in  the  ANOVAs  that  were  also  significant  in 
the  MANOVA  were  reported.  This  was  done  beeause  individual  ANOVAs  do  not  provide  ade¬ 
quate  proteetion  against  making  Type  I  errors  (i.e.,  when  a  true  null  hypothesis  is  rejeeted),  when 
multiple  dependent  variables  are  analyzed  separately.  Performing  the  MANOVA  first  ensures  that 
if  signifieant  differenees  are  found  between  population  means,  the  researeher  may  be  eonfident 
that  real  differenees  aetually  exist  and  ANOVAs  ean  then  be  used  to  determine  where  the 
differenees  aetually  oeeur  (Johnson,  1998). 

To  determine  the  loei  of  signifieanee  in  faetors  with  more  than  two  levels,  a  post  hoc  analysis 
using  the  Tukey-Kramer  Least  Squares  Differenee  test  was  performed  for  eaeh  data  set,  as  appro¬ 
priate.  This  test  was  ehosen  beeause  of  its  ability  to  better  eontrol  error  rate  and  generate  95% 
eonfidenee  intervals  better  than  other  post  hoc  tests,  sueh  as  Newman-Keuls. 

3.1  Overall  Workload 

The  overall  measure  of  workload  was  assessed  aeross  eonditions  (baseline,  CAB,  andNoCAB). 

This  workload  measure  was  an  aggregate  of  the  workload  subseales.  Additionally,  the  individual 
subseales,  ineluding  mental,  physieal,  temporal,  performanee,  effort,  and  frustration,  were  analyzed 
independently  aeross  eonditions.  Presentation  order  was  a  between-subjeet  faetor  and  was  analyzed 
to  determine  whether  the  order  of  eondition  (i.e.,  CAB  first  or  NoCAB  first)  signifieantly  affeeted 
workload  results. 
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For  the  dependent  variable  Overall  Workload,  the  three  levels  in  the  overall  model  were  Baseline 
Workload,  CAB  Workload,  andNoCAB  Workload.  Means  for  this  data  set  are  shown  in  table  1. 
Signifieanee  (at p  <  0.05)  was  observed  for  Workload  in  the  overall  MANOVA  model  using  the 
Wilk’s  lambda  (A,)  test  (see  table  2).  No  signifieant  effect  was  observed  in  the  MANOVA  for 
Order  or  Order  by  Condition. 

ANOVA  results  also  showed  a  significant  effect  for  Overall  Workload  (F2,22=  6.016, =  0.005). 
The  ANOVA  summary  table  for  Overall  Workload  is  provided  in  table  3.  Post  hoc  results  are 
shown  in  table  4.  As  these  results  indicate,  significant  difference  was  observed  between  Baseline 
and  NoCAB  and  between  CAB  and  NoCAB.  For  the  Baseline  and  NoCAB  comparison,  results 
indicate  that  overall  workload  was  significantly  higher  in  the  NoCAB  condition  (42.75)  than  in 
the  Baseline  condition  (30.83).  Also,  results  indicate  that  overall  workload  was  significantly 
higher  in  the  NoCAB  condition  (42.75)  than  in  the  CAB  condition  (33.33).  There  was  no  signi¬ 
ficant  difference  between  the  Baseline  and  CAB  conditions.  Figure  3  provides  a  graphical 
comparison  between  each  of  these  conditions. 


Table  1.  Means  for  overall  workload. 


Workload 

Mean 

Standard 

Error 

95%  Confidence  Interval 

Lower  Bonndary 

Upper  Bonndary 

Baseline 

30.833 

3.782 

22.510 

39.157 

CAB 

33.333 

5.303 

21.661 

45.006 

NoCAB 

42.750 

5.264 

31.163 

54.337 

Table  2.  MANOVA  for  overall  workload. 


Effect 

Valne 

F 

Hypothesis 

df 

Error 

df 

Sig. 

WKLOAD 

Wilks’  lambda  (k) 

.465 

5.181 

2.000 

9.000 

.032* 

WKLOAD  X  ORDER 

Wilks’  lambda  (k) 

.670 

2.220 

2.000 

9.000 

.165 

*Statistically  significant  effect  at/>  <  0.05. 


Tables.  ANOVA  for  overall  workload. 


Sonrce 

Type  III  Snm  of  Sqnares 

df 

Mean 

Sqnare 

F 

Sig. 

WKLOAD 

947.722 

2 

473.861 

7.114 

.005* 

WKLOAD  X  ORDER 

400.722 

2 

200.361 

3.008 

.072 

Error(WKLOAD) 

1332.222 

20 

66.611 

*Statistically  significant  effect  at/>  <  0.05  and  also  significant  in  MANOVA. 
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Table  4.  Tukey-Kramer  comparisons  for  overall  workload-dependent  variables. 


(I)  WKLOAD 

(J)  WKLOAD 

Mean 

Difference 

(I-J) 

Std. 

Error 

Sig. 

95%  Confidence 
Interval  for  Difference 

Lower 

Bonndary 

Upper 

Bonndary 

Baseline 

CAB 

-2.500 

2.957 

.418 

-9.088 

4.088 

NoCAB 

-11.917 

3.643 

.008* 

-20.034 

-3.799 

CAB 

Baseline 

2.500 

2.957 

.418 

-4.088 

9.088 

NoCAB 

-9.417 

3.360 

.019* 

-16.903 

-1.930 

NoCAB 

Baseline 

11.917 

3.643 

.008* 

3.799 

20.034 

CAB 

9.417 

3.360 

.019* 

1.930 

16.903 

*Statistically  significant  effect  aXp<  0.05. 


Lethality  Experiment  Average  Overall  Workload 
(subjective  rating) 


T3  O 
re  <^1 
O 

2  o 
o  (O 

re 


O) 

c 

re 


Condition 


Figure  3.  Comparison  of  overall  workload  across  conditions.  (Means  with  different  letters  are 
significantly  different  atp<  0.05.) 


3.2  Workload  Subscales 

As  previously  stated,  the  NASA- TLX  Workload  Rating  Scale  provides  for  an  overall  workload 
score  as  well  as  a  set  of  specific  subscales  of  workload.  Following  are  the  results  from  these 
subscales.  Presentation  order  was  a  between-subject  factor  and  was  analyzed  to  determine  whether 
the  order  of  condition  (i.e.,  CAB  first  or  NoCAB  first)  significantly  affected  workload. 

The  dependent  variables  included  in  the  Workload  Subscales  model  were  Mental,  Physical, 
Temporal,  Performance,  Effort,  and  Frustration,  in  each  of  three  conditions:  Baseline,  CAB,  and 
NoCAB.  Means  for  this  data  set  are  shown  in  table  5.  Significance  (at p  <  0.05)  was  observed 
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for  Workload  Subscales  in  the  overall  MANOVA  model  with  the  use  of  the  Wilk’s  A,  test  (see 
table  6).  No  significant  effect  was  observed  in  the  MANOVA  for  Order  or  Order  by  Condition. 

ANOVA  results  showed  a  significant  effect  for  the  Mental  (F2,2o=  8.321, =  0.002),  Physical 
(F2,20  =  3.5  03, />  =  0.050),  Temporal  (F2,2o=  lA61,p  =  0.004),  Performanee  (F2,2o=  3.531, 
p  =  0.049),  Effort  (F2,2o=  4.907, />  =  0.018),  and  Frustration  (F2,20  =  3.854, />  =  0.038)  Workload 
Subseales.  The  ANOVA  summary  table  for  Workload  Subseales  is  provided  in  table  7.  Further 
analysis  via  Tukey-Kramer  reveals  that  only  the  Mental,  Temporal,  and  Effort  subseales  exhibited 
significant  differences  when  eompared  across  conditions  (see  table  8). 

Table  5.  Means  for  workload  subscales. 


Workload 

Sub  scale 

Order 

Mean 

Std. 

Error 

95%  Confidence 
Interval  for  Difference 

Lower 

Boundary 

Upper 

Boundary 

MENTAL 

CAB  First 

4.500 

1.306 

1.590 

7.410 

NoCAB  First 

7.500 

1.306 

4.590 

10.410 

PHYSICAL 

CAB  First 

2.556 

1.363 

-.481 

5.592 

NoCAB  First 

5.500 

1.363 

2.464 

8.536 

TEMPORAL 

CAB  First 

3.167 

1.204 

.485 

5.849 

NoCAB  First 

6.944 

1.204 

4.263 

9.626 

PERFORMANCE 

CAB  First 

6.556 

1.526 

3.154 

9.957 

NoCAB  First 

3.056 

1.526 

-.346 

6.457 

EFFORT 

CAB  First 

4.611 

2.051 

.041 

9.181 

NoCAB  First 

11.278 

2.051 

6.708 

15.848 

FRUSTRATION 

CAB  First 

2.556 

.850 

.662 

4.450 

NoCAB  First 

3.722 

.850 

1.828 

5.616 

Table  6.  MANOVA  for  workload  subscales. 


Effect 

Value 

F 

Hypothesis 

df 

Error 

df 

Sig. 

WKLOAD  SUBSCALE 

Pillai’s  trace 

1.157 

3.661 

12.000 

32.000 

.002 

Wilks’  X 

.159 

3.767 

12.000 

30.000 

.002* 

Hotelling’s  trace 

3.296 

3.846 

12.000 

28.000 

.002 

Roy’s  largest  root 

2.502 

6.673 

6.000 

16.000 

.001 

ORDER 

Pillai’s  trace 

.619 

1.356 

6.000 

5.000 

.378 

Wilks’  X 

.381 

1.356 

6.000 

5.000 

.378 

Hotelling’s  trace 

1.627 

1.356 

6.000 

5.000 

.378 

Roy’s  largest  root 

1.627 

1.356 

6.000 

5.000 

.378 

WKLOAD  SUBSCALE 

X  ORDER 

Pillai’s  trace 

.601 

1.146 

12.000 

32.000 

.361 

Wilks’  X 

.478 

1.118 

12.000 

30.000 

.383 

Hotelling’s  trace 

.929 

1.084 

12.000 

28.000 

.409 

Roy’s  largest  root 

.691 

1.842 

6.000 

16.000 

.154 

*Statistically  significant  effect  at/>  <  0.05. 
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Table  7.  ANOVA  for  workload  subscales. 


Source 

Measure 

Type  III  Sum 
of  Squares 

df 

Meau 

Square 

F 

Sig. 

WKLOAD 

MENTAL 

I8I.500 

2 

90.750 

8.351 

.002* 

PHYSICAL 

31.722 

2 

15.861 

3.503 

.050* 

TEMPORAL 

62.889 

2 

31.444 

7.467 

.004* 

PERFORMANCE 

44.056 

2 

22.028 

3.531 

.049* 

EFFORT 

35.056 

2 

17.528 

4.907 

.018* 

FRUSTRATION 

48.222 

2 

24.III 

3.854 

.038* 

WKLOAD  SUBSCALE 

X  ORDER 

MENTAL 

11.167 

2 

5.583 

.514 

.606 

PHYSICAL 

14.389 

2 

7.194 

1.589 

.229 

TEMPORAL 

21.556 

2 

10.778 

2.559 

.102 

PERFORMANCE 

45.167 

2 

22.583 

3.620 

.056 

EFFORT 

.167 

2 

.083 

.023 

.977 

FRUSTRATION 

16.667 

2 

8.333 

1.332 

.286 

Error  (WKLOAD 
SUBSCALE) 

MENTAL 

217.333 

20 

10.867 

PHYSICAL 

90.556 

20 

4.528 

TEMPORAL 

84.222 

20 

4.2 1 1 

PERFORMANCE 

124.778 

20 

6.239 

EFFORT 

71.444 

20 

3.572 

FRUSTRATION 

125. Ill 

20 

6.256 

*  Statistically  significant  effect  at/>  <  0.05  and  also  significant  in  MANOVA 


Table  8.  Tukey-Kramer  comparisons  for  workload  subscales. 


WKLOAD 

SUBSCALE 

(I)  COND 

(J)  COND 

Meau 

Differeuce 

(I-J) 

Std. 

Error 

Sig. 

95%  Coufideuce 
luterval  for  Differeuce 

Lower 

Bouudary 

Upper 

Bouudary 

MENTAL 

Baseline 

CAB 

-2.750 

1.539 

.104 

-6.179 

.679 

NoCAB 

-5.500 

1.662 

.008* 

-9.204 

-1.796 

CAB 

Baseline 

2.750 

1.539 

.104 

-.679 

6.179 

NoCAB 

-2.750 

.549 

.001* 

-3.973 

-1.527 

NoCAB 

Baseline 

5.500 

1.662 

.008* 

1.796 

9.204 

CAB 

2.750 

.549 

.001* 

1.527 

3.973 

PHYSICAL 

Baseline 

CAB 

.417 

.523 

.444 

-.749 

1.582 

NoCAB 

-1.750 

.797 

.053 

-3.525 

.025 

CAB 

Baseline 

-.417 

.523 

.444 

-1.582 

.749 

NoCAB 

-2.167 

1. 164 

.092 

-4.761 

.428 

NoCAB 

Baseline 

1.750 

.797 

.053 

-.025 

3.525 

CAB 

2.167 

1. 164 

.092 

-.428 

4.761 

TEMPORAL 

Baseline 

CAB 

-1.000 

.745 

.209 

-2.661 

.661 

NoCAB 

-3.167 

1.078 

.015* 

-5.568 

-.766 

CAB 

Baseline 

1. 000 

.745 

.209 

-.661 

2.661 

NoCAB 

-2.167 

.624 

.006* 

-3.556 

-.777 

NoCAB 

Baseline 

3.167 

1.078 

.015* 

.766 

5.568 

CAB 

2.167 

.624 

.006* 

.777 

3.556 

PERFORMANCE 

Baseline 

CAB 

2.583 

1.200 

.057 

-.091 

5.257 

NoCAB 

2.000 

1.098 

.099 

-.446 

4.446 

CAB 

Baseline 

-2.583 

1.200 

.057 

-5.257 

.091 

NoCAB 

-.583 

.688 

.416 

-2.117 

.950 

NoCAB 

Baseline 

-2.000 

1.098 

.099 

-4.446 

.446 

CAB 

.583 

.688 

.416 

-.950 

2.117 
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EFFORT 

Baseline 

CAB 

1.167 

.833 

.192 

-.690 

3.023 

NoCAB 

-1.250 

.827 

.162 

-3.094 

.594 

CAB 

Baseline 

-1.167 

.833 

.192 

-3.023 

.690 

NoCAB 

-2.417 

.638 

.004* 

-3.838 

-.995 

NoCAB 

Baseline 

1.250 

.827 

.162 

-.594 

3.094 

CAB 

2.417 

.638 

.004* 

.995 

3.838 

FRUSTRATION 

Baseline 

CAB 

-.500 

.350 

.183 

-1.279 

.279 

NoCAB 

-2.667 

1.256 

.060 

-5.465 

.132 

CAB 

Baseline 

.500 

.350 

.183 

-.279 

1.279 

NoCAB 

-2.167 

1.195 

.100 

-4.829 

.496 

NoCAB 

Baseline 

2.667 

1.256 

.060 

-.132 

5.465 

CAB 

2.167 

1.195 

.100 

-.496 

4.829 

*Statistically  significant  effect  at/>  <  0.05. 


3,2.1  Mental  Workload 

For  Mental  Workload,  a  significant  difference  was  observed  between  Baseline  and  NoCAB  and 
between  CAB  and  NoCAB.  For  the  Baseline  and  NoCAB  comparison,  results  indicate  that  mental 
workload  was  significantly  higher  in  the  NoCAB  condition  (8.75)  than  in  the  Baseline  condition 
(3.25).  Similarly,  results  indicate  that  mental  workload  was  significantly  higher  in  the  NoCAB 
condition  (8.75)  than  in  the  CAB  condition  (6.0).  There  was  no  significant  difference  between  the 
Baseline  and  CAB  conditions  for  Mental  Workload.  Figure  4  provides  a  graphical  comparison 
between  each  of  these  conditions  for  Mental  Workload. 


3.2.2  Temporal  Workload 

For  Temporal  Workload,  a  significant  difference  was  observed  between  Baseline  and  NoCAB 
and  between  CAB  and  NoCAB.  For  the  Baseline  and  NoCAB  comparison,  results  indicate  that 
temporal  workload  was  significantly  higher  in  the  NoCAB  condition  (6.83)  than  in  the  Baseline 
condition  (3.67).  Results  indicate  that  temporal  workload  was  significantly  higher  in  the  NoCAB 
condition  (6.83)  than  in  the  CAB  condition  (4.67).  There  was  no  significant  difference  between 
the  Baseline  and  CAB  conditions  for  Temporal  Workload.  Figure  5  provides  a  graphical 
comparison  between  each  of  these  conditions  for  temporal  workload. 

3.2.3  Effort  Workload 

For  Effort  Workload,  a  significant  difference  was  observed  only  between  CAB  and  NoCAB. 
Results  indicate  that  effort  workload  was  significantly  higher  in  the  NoCAB  condition  (9.17)  than 
in  the  CAB  condition  (6.75).  Although  effort  workload  was  lower  in  the  CAB  condition  than  in 
the  Baseline  condition,  this  difference  was  not  significant.  Figure  6  provides  a  graphical 
comparison  between  each  of  these  conditions  for  Effort  Workload. 

Eigure  7  provides  a  graphical  comparison  between  all  Workload  Subscales  across  the  three 
conditions.  Although  only  the  Mental,  Temporal,  and  Effort  subscales  are  significant,  this  figure 
clearly  shows  the  trend  of  increasing  workload  for  all  subscales  in  the  NoCAB  condition  when 
compared  with  the  Baseline  and  CAB  conditions.  Also,  it  can  be  seen  that  effort  is  reduced  when 
CABs  were  used,  compared  with  the  Baseline  and  NoCAB  conditions.  Einally,  this  graph  shows 
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Workload  Rating  ^  Workload  Rating 


that  participants  perceived  that  their  performanee  was  inereased  slightly  when  they  used  the 
CABs,  eompared  to  when  they  did  not  use  the  CABs  (note  that  performanee  is  a  reverse  seale). 


Lethality  Experiment  Average  Workload 

(Mental) 


Baseline  CAB  NoCAB 

Condition 


4.  Comparison  of  mental  workload  across  conditions.  (Means  with  different  letters  are 
significantly  different  at/?  <  0.05.) 


Lethality  Experiment  Average  Workload 
(Temporal) 


Baseline  CAB  NoCAB 

Condition 

Figure  5.  Comparison  of  temporal  workload  across  conditions.  (Means  with  different  letters 
are  significantly  different  at/?  <  0.05.) 
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Lethality  Experiment  Average  Workload 
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Figure  6.  Comparison  of  temporal  workload  across  conditions.  (Means  with  different  letters  are 
significantly  different  atp<  0.05.) 


Lethality  Experiment  Average  Workload 
(Subscale  Comparison) 


Figure  7.  Workload  subscale  comparison  across  conditions  (performance  is  a  reverse  scale). 


3,3  Stress 

Mental  stress  and  physieal  stress  data  were  evaluated  aeross  eonditions.  Comparisons  between 
mental  and  physical  stress  within  each  condition  were  not  conducted  since  differences  between 
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the  two  were  expeeted  but  not  relevant  to  this  effort.  Presentation  order  was  a  between-subjeet 
faetor  and  was  analyzed  to  determine  whether  the  order  of  eondition  (i.e.,  CAB  first  or  NoCAB 
first)  signifieantly  affeeted  stress. 

For  the  dependent  variable  Mental  Stress,  the  three  levels  in  the  overall  model  were  Baseline 
Mental  Stress,  CAB  Mental  Stress,  and  NoCAB  Mental  Stress.  Means  for  this  data  set  are  shown 
in  table  9.  No  statistieal  significanee  (at  p  <  0.05)  was  observed  for  Mental  Stress  in  the  overall 
MANOVA  model  using  the  Wilk’s  A,  test  (see  table  10).  Inspeetion  of  the  ANOVA  for  Mental 
Stress  also  showed  no  signifioanee. 

For  the  dependent  variable  Physieal  Stress,  the  three  levels  in  the  overall  model  were  Baseline 
Physieal  Stress,  CAB  Physieal  Stress,  and  NoCAB  Physical  Stress.  Means  for  this  data  set  are 
shown  in  table  1 1 .  Significance  (at p  <  0.05)  was  observed  for  Physical  Stress  in  the  overall 
MANOVA  model  using  the  Wilk’s  A,  test  (see  table  12).  No  significant  effect  was  observed  in 
the  MANOVA  for  Order  or  Order  by  Condition. 

Although  MANOVA  results  for  Physical  Stress  showed  significance  in  the  overall  model,  a 
closer  inspection  of  the  data  via  the  ANOVA  showed  no  significant  effect  for  Physical  Stress 
(F2,22  =  3.462,  p  =  0.05 1),  albeit  just  slightly  non-significant.  The  ANOVA  summary  table  for 
Physical  Stress  is  provided  in  table  13.  Although  statistical  significance  was  not  observed  across 
conditions  for  physical  or  mental  stress,  figure  8  is  provided  to  show  the  general  trend  of  increas¬ 
ing  stress  (both  physical  and  mental)  in  the  NoCAB  condition  when  compared  with  the  Baseline 
and  CAB  conditions.  It  is  also  shown  that  stress  (both  physical  and  mental)  is  only  slightly 
elevated  in  the  CAB  condition  when  compared  to  the  Baseline  condition. 

Table  9.  Means  for  mental  stress. 


MSTRESS 

Mean 

Standard 

Error 

95%  Confidence  Interval 

Lower  Bonndary 

Upper  Bonndary 

Baseline 

2.000 

.246 

1.458 

2.542 

CAB 

2.083 

.288 

1.450 

2.716 

NoCAB 

2.667 

.333 

1.933 

3.400 

Table  10.  MANOVA  for  mental  stress. 


Effect 

Valne 

F 

Hypothesis 

df 

Error 

df 

Sig. 

MSTRESS 

Pillai’s  trace 

.359 

2.516 

2.000 

9.000 

.136 

Wilks’  X 

.641 

2.516 

2.000 

9.000 

.136 

Hotelling’s  trace 

.559 

2.516 

2.000 

9.000 

.136 

Roy’s  largest  root 

.559 

2.516 

2.000 

9.000 

.136 

MSTRESS  X  ORDER 

Pillai’s  trace 

.229 

1.339 

2.000 

9.000 

.310 

Wilks’  X 

.111 

1.339 

2.000 

9.000 

.310 

Hotelling’s  trace 

.298 

1.339 

2.000 

9.000 

.310 

Roy’s  largest  root 

.298 

1.339(a) 

2.000 

9.000 

.310 

*Statistically  significant  effect  at/>  <  0.05. 
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Table  11.  Means  for  physical  stress. 


PSTRESS 

Mean 

Standard 

Error 

95%  Confidence  Interval 

Lower  Bonndary 

Upper  Bonndary 

Baseline 

1.667 

.284 

1.041 

2.292 

CAB 

1.667 

.225 

1.172 

2.161 

NoCAB 

2.167 

.322 

1.458 

2.875 

Table  12.  MANOVA  for  physical  stress. 


Effect 

Valne 

F 

Hypothesis 

df 

Error 

df 

Sig. 

PSTRESS 

Pillai’s  trace 

.529 

5.063 

2.000 

9.000 

.034 

Wilks’  X 

.471 

5.063 

2.000 

9.000 

.034* 

Flotelling’s  trace 

1.125 

5.063 

2.000 

9.000 

.034 

Roy’s  largest  root 

1.125 

5.063 

2.000 

9.000 

.034 

PSTRESS  X  ORDER 

Pillai’s  trace 

.111 

.563 

2.000 

9.000 

.589 

Wilks’  X 

.889 

.563 

2.000 

9.000 

.589 

Flotelling’s  trace 

.125 

.563 

2.000 

9.000 

.589 

Roy’s  largest  root 

.125 

.563 

2.000 

9.000 

.589 

*Statistically  significant  effect  at/>  <  0.05. 


Tablets.  ANOVA  for  physical  stress. 


Source 

Type  III  Snm  of 
Squares 

df 

Mean 

Square 

F 

Sig. 

PSTRESS 

2.000 

2 

1.000 

3.462 

.051 

PSTRESS  X  ORDER 

.222 

2 

.111 

.385 

.686 

Error(PSTRESS) 

5.778 

20 

.289 

*Statistically  significant  effect  at/>  <  0.05  and  also  significant  in  MANOVA. 
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Figure  8.  Comparison  of  physical  and  mental  stress  across  conditions.  (Means  with  different 
letters  are  significantly  different  at/?  <  0.05.) 
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3,4  Situation  Awareness 


The  dependent  variable  ineluded  in  the  overall  model  was  SA.  The  three  levels  of  SA  were 
Baseline  SA,  CAB  SA,  and  NoCAB  SA.  Presentation  order  was  a  between-subjeet  faetor  and  was 
analyzed  to  determine  whether  the  order  of  eondition  (i.e.,  CAB  first  or  NoCAB  first)  signifieantly 
affeeted  SA.  Means  for  this  data  set  are  shown  in  table  14.  No  statistieal  signifioanee  (at p  <  0.05) 
was  observed  for  SA  in  the  overall  MANOVA  model  using  the  Wilk’s  X  test  (see  table  15).  No 
signifieant  effeet  was  observed  in  the  MANOVA  for  Order  or  Order  by  Condition. 

Although  statistieal  signifieanee  was  not  observed  aeross  eonditions  for  SA,  figure  9  is  provided 
to  show  the  trend  of  deereasing  SA  in  the  NoCAB  eondition  when  eompared  with  the  Baseline 
and  CAB  eonditions  and  to  show  that  SA  is  slightly  inereased  in  the  CAB  eondition  when 
eompared  to  the  Baseline  eondition. 

Table  14.  Means  for  situation  awareness. 


SITUATION 

AWARENESS 

Mean 

Standard 

Error 

95%  Confidence  Interval 

Lower  Bonndary 

Upper  Bonndary 

Baseline 

9.000 

.337 

8.248 

9.752 

CAB 

9.167 

.284 

8.534 

9.799 

NoCAB 

8.583 

.271 

7.979 

9.188 

Table  15.  MANOVA  for  situation  awareness. 


Effect 

Value 

F 

Hypothesis 

df 

Error 

df 

Sig. 

SITUATION 

AWARENESS 

Pillai’s  trace 

.459 

3.813 

2.000 

9.000 

.063 

Wilks’  X 

.541 

3.813 

2.000 

9.000 

.063 

Hotelling’s  trace 

.847 

3.813 

2.000 

9.000 

.063 

Roy’s  largest  root 

.847 

3.813 

2.000 

9.000 

.063 

SITUATION  AWARE¬ 
NESS  X  ORDER 

Pillai’s  trace 

.301 

1.940 

2.000 

9.000 

.199 

Wilks’  X 

.699 

1.940 

2.000 

9.000 

.199 

Hotelling’s  trace 

.431 

1.940 

2.000 

9.000 

.199 

Roy’s  largest  root 

.431 

1.940 

2.000 

9.000 

.199 

*Statistically  significant  effect  at/>  <  0.05. 
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Lethality  Experiment  Average  Situation  Awareness 
(subjective  rating) 


Condition 

Figure  9.  Comparison  of  situation  awareness  across  conditions.  (Means  with  different  letters  are 
significantly  different  at /?  <  0.05.) 

3.5  Task  Time 

The  dependent  variable  ineluded  in  the  overall  model  was  Task  Time.  The  two  levels  of  Task 
Time  were  CAB  Task  Time  and  NoCAB  Task  Time.  Presentation  order  was  a  between-subjeet 
faetor  and  was  analyzed  to  determine  whether  the  order  of  eondition  (i.e.,  CAB  first  or  NoCAB 
first)  signifieantly  affeeted  task  time.  Means  for  this  data  set  are  shown  in  table  16.  Signifieanee 
(at  p  <  0.05)  was  observed  for  Task  Time  and  the  interaetion  of  Task  Time  and  Presentation 
Order  in  the  overall  MANOVA  model  with  the  Wilk’s  A,  test  (see  table  17). 

ANOVA  results  also  showed  a  signifieant  effeet  for  Task  Time  (Fi,io=  66.076, <  0.0001)  and 
the  interaetion  of  Task  Time  and  Presentation  Order  (F2,22  =  7.707,  p  =  0.020).  The  ANOVA 
summary  table  for  Task  Time  is  provided  in  table  18.  Post  hoc  results  were  not  eondueted  sinee 
there  were  only  two  levels  of  the  independent  variable.  To  determine  the  loei  of  signifioanee 
between  the  two  levels  of  task  time  (shown  to  be  signifieant)  and  between  the  two  levels  of 
presentation  order  (also  shown  to  be  signifieant),  the  means  were  examined.  Interaetion  means 
are  shown  in  table  19. 

Figure  10  shows  the  average  time  to  eomplete  the  task  in  both  eonditions.  As  stated,  results 
showed  a  signifieant  differenee  between  eonditions  for  time  to  eomplete  the  task.  Partieipants 
took  signifieantly  less  time  to  eomplete  the  task  in  the  CAB  eondition  (5.32  minutes)  than  in  the 
NoCAB  eondition  (8.26  minutes). 

Figure  1 1  shows  the  signifieant  interaetion  effeet  between  eondition  and  presentation  order. 
Results  from  the  interaetion  means  table  (table  19)  and  figure  1 1  indieate  that  in  the  NoCAB 
eondition,  when  NoCAB  was  presented  first,  partieipants  took  signifieantly  longer  time  to 
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complete  the  task  (9.13  minutes)  than  those  partieipants  who  were  presented  CAB  first  (7.38). 
Conversely,  time  to  eomplete  the  task  in  the  CAB  eondition  was  not  signifieantly  different, 
regardless  whieh  eondition  was  presented  first  (i.e.,  5.45  minutes  with  CAB  first  versus  5.18 
minutes  with  NoCAB  first). 

3,5.1  Engagement-only  Analysis 

To  assess  the  potential  effect  or  bias  on  the  engagement-only  sub  task,  a  seeondary  analysis  was 
eonducted  only  on  the  targets  engaged  by  the  Soldiers.  This  data  set  did  not  inelude  the  prioritiza¬ 
tion  or  weapon  and  munition  seleetion  subtasks  and  ineluded  only  those  targets  engaged  by  all 
Soldiers.  That  is,  some  Soldiers  did  not  engage  some  targets  beeause  ammunition  was  depleted 
before  all  targets  eould  be  engaged  (caused  by  multiple  shots  on  the  same  targets  beeause  of 
system  fidelity  problems).  When  these  targets  (a  total  of  three)  were  removed  from  the  data  set 
and  the  remaining  data  were  analyzed,  it  was  shown  that  engagement-only  times,  that  is,  without 
the  task  of  prioritizing  targets,  were  not  signifieantly  different  aeross  eonditions  (see  figure  12). 
This  was  expeeted  sinee  target  engagement  should  not  have  been  affeeted  by  eondition. 

Results  from  this  seeondary  analysis  are  only  reported  here  to  further  illustrate  the  effeet  of  the 
CABs  in  the  prioritization  analysis  beeause  this  test  eliminates  the  potential  effect  of  engagement 
time  from  the  prioritization  time.  That  is,  the  task  of  seleeting  and  shooting  targets  should  have 
been  (and  was)  the  same  in  eaeh  eondition  and  should  not  have  been  infiueneed  (and  was  not)  by 
the  CABs  or  absenee  of  CABs.  That  is,  it  ean  be  stated  that  the  significant  differences  between 
conditions  in  the  prioritization  portion  of  the  task  are  unambiguous  in  that  the  observed  differenees 
were  eaused  by  the  faetors  of  interest  and  were  in  no  way  biased  by  an  engagement  effeet. 


Table  16.  Means  for  task  time. 


TASK  TIME 

Mean 

Standard 

Error 

95%  Confidence  Interval 

Lower  Bonndary 

Upper  Bonndary 

CAB 

318.917 

9.774 

297.138 

340.696 

NoCAB 

495.333 

19.972 

450.834 

539.833 

Table  17.  MANOVA  for  task  time. 


Effect 

Valne 

F 

Hypothesis 

df 

Error 

df 

Sig. 

TASK  TIME 

Pillai’s  trace 

.869 

66.076 

1.000 

10.000 

.000 

Wilks’  X 

.131 

66.076 

1.000 

10.000 

.000* 

Hotelling’s  trace 

6.608 

66.076 

1.000 

10.000 

.000 

Roy’s  largest  root 

6.608 

66.076 

1.000 

10.000 

.000 

TASK  TIME  X  ORDER 

Pillai’s  trace 

.435 

7.707 

1.000 

10.000 

.020 

Wilks’  X 

.565 

7.707 

1.000 

10.000 

.020* 

Hotelling’s  trace 

.111 

7.707 

1.000 

10.000 

.020 

Roy’s  largest  root 

.111 

7.707 

1.000 

10.000 

.020 

*  Statistically  significant  effect  at/)  <  0.05. 
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Table  18.  ANOVA  for  task  time. 


Source 

Type  III  Sum  of 
Squares 

df 

Meau 

Square 

F 

Sig. 

TASKTIME 

373474.083 

1 

373474.083 

66.076 

.000* 

TASKTIME  X  ORDER 

43560.750 

1 

43560.750 

7.707 

.020* 

Error(TASKTIME) 

56522.167 

10 

5652.217 

*Statistically  significant  effect  at/)  <  0.05  and  also  significant  in  MANOVA. 


Table  19.  Means  for  task  time  and  presentation  order. 


ORDER 

TASK  TIME 

Meau 

Staudard 

Error 

95%  Coufideuce  luterval 

Lower  Bouudary 

Upper  Bouudary 

CAB  First 

CAB 

326.833 

13.823 

296.033 

357.633 

NoCAB 

443.000 

28.244 

380.068 

505.932 

NoCAB  First 

CAB 

311.000 

13.823 

280.200 

341.800 

NoCAB 

547.667 

28.244 

484.735 

610.598 

Lethality  Experiment  Average  Time 

to  Complete  Task 

(target  prioritization  and  engagement) 


CAB 

c 

.2 

c 
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^  NoCAB 


0  2  4  6  8  10 

Time  (minutes) 


? 

(A)  5.3: 

- 

(B)  8.26 

Figure  10.  Average  time  to  complete  task.  (Means  with  different  letters  are  significantly 
different  at/?  <  0.05.) 
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Figure  1 1 .  Condition  x  Presentation  Order  interaetion.  (Means  with  different  letters  are  signifieantly 
different  at/?  <  0.05.) 


Figure  12.  Average  engagement-only  times.  (Means  with  different  letters  are  significantly  different 
at/?  <  0.05.) 
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4.  Discussion 


4,1  Workload 

Results  of  this  study  show  that  CABs  that  aid  the  Soldier  in  prioritizing  targets  and  seleeting  the 
appropriate  weapons  platform  and  munition  type  have  a  significant  impact  on  overall  Soldier 
workload.  Specifically,  overall  workload  was  significantly  reduced  when  Soldiers  conducted 
target  prioritization  and  engagement  tasks  using  CABs.  Conversely,  overall  workload  was 
significantly  increased,  compared  with  baseline  and  CAB  conditions,  when  Soldiers  conducted  the 
same  tasks  without  using  CABs.  The  CABs  provided  the  Soldiers  with  a  pre-sorted  list  of  known 
targets,  their  threat  potential,  and  their  proximity.  This  information  was  then  automatically 
matched  with  an  appropriate  weapons  platform  and  munition  type  that  was  recommended  to  the 
Soldier  for  use  in  target  engagement.  In  the  absence  of  these  decision  aids.  Soldiers  scrolled 
through  and  visually  scanned  the  list  of  known  targets,  assessed  each  for  threat,  proximity,  and 
appropriate  weapon  and  munition  type  for  engagement,  and  then  kept  track  of  this  information 
mentally  while  they  conducted  a  mental  sort,  prioritization,  and  recommendation  procedure. 

As  stated,  the  workload  subscale  results  showed  a  significant  difference  between  CAB  and 
NoCAB  for  mental,  temporal,  and  effort  workload.  For  mental  workload,  significant  difference 
between  conditions  means  that  without  CABs,  a  significantly  greater  mental  demand  was  placed 
on  the  Soldier  during  the  task.  That  is.  Soldiers  were  required  to  look,  search,  think,  calculate  and 
remember  significantly  more  than  when  they  used  the  CABs  to  complete  the  same  task.  Using 
CABs  allowed  the  Soldiers  to  perform  their  task  while  significantly  reducing  their  mental 
workload.  Similarly,  CABs  significantly  reduced  Soldier’s  temporal  workload,  when  compared 
with  the  NoCAB  condition.  This  means  that  Soldiers  did  not  feel  significant  time  pressure  while 
performing  their  task  when  using  CABs.  However,  when  they  did  not  use  CABs,  Soldiers  felt 
significant  time  pressure  to  complete  their  task.  This  pressure  contributed  to  the  overall  workload 
experienced  when  Soldiers  did  not  use  CABs.  CABs  significantly  reduced  the  Soldiers’  effort 
workload  when  compared  with  the  NoCAB  condition.  This  means  that  using  CABs  to  perform 
their  task  significantly  reduced  the  effort  required  to  prioritize  and  engage  known  targets.  Without 
CABs,  Soldiers  felt  they  needed  to  work  significantly  harder  (i.e.,  exert  greater  effort)  to  prioritize 
and  engage  the  targets. 

Although  only  the  mental,  temporal,  and  effort  workload  subscales  were  significant,  all  workload 
subscales  showed  the  general  trend  of  increasing  workload  for  all  subscales  in  the  NoCAB 
condition  when  compared  with  the  Baseline  and  CAB  conditions.  It  is  believed  that  a  larger  study 
with  a  larger  sample  size  would  show  that  this  trend  continues  to  increase  and  would  show  a 
significant  effect  across  all  workload  subscales.  Results  also  showed  that  participants  perceived 
their  performance  was  increased  slightly  when  they  used  the  CABs,  compared  to  when  they  did  not 
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use  the  CABs.  Again,  with  a  larger  sample  size,  it  is  believed  this  trend  would  also  beeome 
statistically  significant. 

4.2  Stress 

As  the  results  indicated,  physical  stress  was  only  slightly  non-signilicant.  Even  though  no  signifi¬ 
cance  was  observed,  the  general  trend  of  increasing  stress  (both  mental  and  physical)  may  be  seen 
between  CAB  and  NoCAB.  That  is,  both  mental  and  physical  stress  are  higher  in  the  NoCAB 
condition  than  in  the  CAB  or  Baseline  conditions.  Although  this  is  not  statistically  significant,  it  is 
believed  this  difference  would  become  so  with  a  larger  sample  size. 

4.3  Task  Time 

Results  showed  a  significant  reduction  in  the  amount  of  time  required  to  complete  the  task  when 
CABs  were  used  versus  when  they  were  not.  Specifically,  Soldiers  performed  the  task  36%  faster 
when  using  CABs.  This  result  is  primarily  because  sorting,  prioritizing,  and  weapon  and  munition 
assignment  are  conducted  automatically  in  the  CAB  condition.  In  this  condition.  Soldiers  merely 
needed  to  review  the  selection  and  recommendation  before  engagement.  Without  CABs,  however. 
Soldiers  were  required  to  scroll  through  the  list  of  known  targets,  assess  each  for  threat,  proximity, 
and  appropriate  weapon  and  munition  type,  and  then  keep  track  of  this  information  mentally  while 
they  conducted  a  mental  sort,  prioritization,  and  recommendation  procedure.  This  NoCAB 
condition  took  significantly  greater  time  (8.26  minutes  without  CABs  versus  5.32  minutes  with 
CABs). 

The  significant  Condition  x  Order  interaction  suggests  the  presence  of  a  learning  effect.  That  is. 
Soldiers  in  the  NoCAB  condition  took  less  time  to  prioritize  targets  when  the  CAB  condition 
was  run  first,  versus  second.  This  suggests  that  Soldiers  “learned”  from  the  way  the  CAB 
prioritized  targets  and  used  that  knowledge  to  make  their  own  prioritization  faster  in  the  NoCAB 
condition.  There  was  no  learning  effect  in  the  CAB  conditions.  Although  learning  effect  is  an 
unintended  artifact,  which  was  controlled  by  the  counterbalancing  of  treatment  order,  it  works  in 
favor  of  the  hypothesis  that  CABs  aid  performance.  That  is,  without  this  learning  effect,  times  in 
the  NoCAB  condition  where  CABs  were  run  first  would  have  been  even  longer,  thus  increasing 
the  already  large  difference  between  CAB  and  NoCAB  task  times.  In  other  words,  without  the 
learning  effect,  the  positive  effect  of  CABs,  which  is  already  significantly  greater  than  NoCABs, 
would  be  even  greater. 
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5.  Conclusion 


Results  of  this  study  have  shown  that  CABs  signifieantly  reduee  time  and  workload  when  Soldiers 
are  eondueting  the  task  of  prioritizing  and  engaging  targets.  Soldiers  took  signifieantly  less  time 
to  eomplete  the  prioritization  and  engagement  task  when  using  CABs  versus  when  they  performed 
the  same  task  manually  (i.e.,  the  NoCAB  eondition).  Overall  task  time  was  redueed  by  36%  when 
CABs  were  used. 

Overall  workload,  as  well  as  the  subseales  of  mental,  temporal,  and  effort  workload,  were  signifi- 
eantly  redueed  when  CABs  were  used.  Overall  workload  was  28%  less  when  CABs  were  used 
versus  when  they  were  not.  Mental  and  temporal  workload  were  both  46%  less  when  CABs  were 
used  versus  when  they  were  not,  and  effort  workload  was  36%  less  when  CABs  were  used  versus 
when  they  were  not. 

For  those  performanee  measures  that  were  not  shown  to  be  signifieant  (i.e.,  situation  awareness, 
stress,  and  the  remaining  workload  subseales),  results  indieate  that  these  measures  exhibit  the 
general  trend  of  favoring  the  CAB  eondition  versus  the  NoCAB  eondition.  For  example,  both 
mental  and  physieal  stress,  while  only  slightly  non-signifieant,  exhibit  a  generally  deereasing  trend 
in  the  CAB  eondition.  Deereasing  stress  is  a  positive  eontribution.  Similarly,  again,  while  only 
slightly  non-signifieant,  situation  awareness  exhibits  a  generally  deereasing  trend  in  the  NoCAB 
eondition  (it  is  only  slightly  inereased  in  the  CAB  eondition).  Deereasing  SA  is  not  a  positive 
eontribution.  Situation  awareness  was  not  deereased  when  CABs  were  used  for  this  task. 

It  is  apparent  from  the  statistieal  results  of  this  study  that  CABs  provide  benefieial  effeet  for 
Soldiers  when  they  are  performing  the  task  of  target  prioritization  and  engagement  as  outlined  in 
this  study.  It  is  believed  that  the  general  trend  observed  in  the  non-signifieant  results  would 
beeome  statistieally  signifieant  if  a  larger  sample  size  were  exposed  to  the  same  eonditions 
outlined  in  this  study. 
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Appendix  A.  SIL  Crew  Station  Layout  and  Functionality  Descriptions 


(from  AMRDEC  training  slides) 


Target  Acquisition  Display  -  Yoke  Controls 


Magnification 

•  Increase  /  decrease  the  TA  sensor 
magnification  from  0.5X  to  24X. 


Slew 

•  Slew  TA  sensor  left  /  right 

•  Tilt  TA  sensor  up  /  down 


Fight  Suite  -  Target  Queue 


Reports 


Target  Acquisition 


Planning 


Assets 


Target  Queue 


Planning 
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Target  Queue  Display 


O  Sort  Target  List 


© 


Highlight  Target 

-  Scroll  thru  queue  to  highlight  targets  by 
using  touch-screen  or  TGT  LIST  button 
on  yoke 


Note:  Mighli^icd  Target  will  have  cross'hairs 
overbid  on  h  on  SA  map 


© 


Select  Target 

-  Select  target  by  depressing  TGT  LIST 
button  on  yoke  (will  slew  the  TA  sensor) 

Target  Details 
Target  Priority 
(s)  Target  Type  &  ID 

Range  from  CAT  (vLAV9)  &  Az 
position  of  Weapon  System 
Q  Weapon  Recommendation  &  Asset 


Move 

Up 


Move 

OOM) 


High  Airnor  £1006 

Range  3676  meters  Azimuth  141  deyees 
Weapon  106  Asset  vLAVd 


High  Airrtor  £1003 

Range  4226  meters  Azimuth  141  degrees 
Weapon  CMS  Asset  vXUV6 


Aimor  £1001 

Range  4577  meters.  Azimuth  141 
Weapon  CMS  Asset  vXUVe 


High  Armor  £1002 

Range  4427  meters  Azimuth  144deyees 
Weapon  CMS  Asset  vXUV6 


Infantry  £1007 

Range  3699  meters.  Azimuth  140  degrees 
Weapon;  LCRK  Asset  vXUV6 


Infantry  £1009 

Rang#  3970  nvaters.  Azimuth  130degraes 
Weapon  LCPK  Asset  vXUV6 


Low 

4 


Infantry  £1005  ^ 

Range  4672  meters  Azimuth  146  degrees  6 
Weapon  None  Asset  None  - 


Vl»w 

lm«o«\ 


OMM* 

T«g*( 


Sort  by 
Piloray 


Scenario  Specific  Strategies 

•  Mission 

-  You  are  a  gunner  operating  as  part  of  3  man  MCS  crew.  You  have  been  Task  Organized 
with  an  ARV-A  and  ARV-R.  And,  you  are  providing  fire  support  for  an  RS&V  Company. 
Your  PLT  LDR  has  detected  a  potential  threat  at  approximately  2.5km  from  your  vehicle 
that  requires  additional  recon  /  surveillance.  He  has  instructed  the  ARV-R  to  investigate. 

-  Your  mission  begins  once  the  ARV-R  recon  is  complete.  Your  mission  is  to  analyze  all 
targets  that  are  available  to  you  via  the  Target  Queue  and  engage  and  defeat  as  many  of 
those  threats  as  possible,  as  quickly  as  possible. 

-  You  will  be  conducting  two  experiment  runs.  The  first  run  will  not  prioritize  any  of  the 
targets,  nor  will  it  provide  any  weapon  recommendations.  This  is  the  no  CABs  condition. 
The  second  run  will  prioritize  targets  and  provide  weapon  recommendations.  This  will  be 
the  CABs  condition.  The  overall  improvements  in  crew  performance  with  the  CABs  will  be 
measured. 

-  Targets  that  you  will  encounter  include: 

•  T-72’s,  which  have  a  maximum  effective  range  of  3.5km 

•  BMP-2's,  equipped  with  Missile  Launchers,  also  have  a  maximum  effective  range  of  3.5km 


26 


Experiment  -  Initial  State 


•  Initialize  OTB  and  SA  Server;  load  the  X4  scenario  in  OTB 


•  Initialize  MC2  (ensure  the  DSS  is  NOT  connected);  load  X4 


•  Initialize  SMI  (CABs  ON  /  OFF) 

-  load  the  X4  scenario 

-  bring  up  the  FIGHT  suite 

-  login  into  vXUV8 

-  adjust  zoom  factor  of  SA  map 


•  Once  all  targets  have  been  spotted,  connect  MC2  with  DSS 

•  Begin  experiment 


Target  Engagement  Sequence 
(no  CABS) 

O  Scroll  thru  target  list;  find  target  to 
engage;  highlight  target 

(D  Seiect  Asset  /  Weapon 

NOTE:  Operator  may  choose  to  override 
rceommcaditioB  with  selection 


0  Seiect  Target  &  zoom  sensor  (if  nec) 
0  ARM  Weapon  System  by  depressing  palm 

grips  on  yoke 

NOTE:  This  will  AIM  the  Weapon  System 


0  Wait  for  Fire  Permit* 


Fire  Weapon  by  pulling  trigger 


Amor  E1003 

Rang*  4220  m««*(«  AfimuOt  141  d*gr*« 
Weapon  CMS  Am*(  vXUVO 


A/mor  E1001 

Rang*  4077  m*4*r»  AiimuOt  141  d 
W**pon  CMS  As*«t  vXUVS 


Amor  E1002 

Rang*  4427  rn*|*ts  AwnuOt  144  d*gr*« 
W**pon  CMS  As««t  vXUVS 


IMBitty  E1007 

Rang*  3000m*4*r«  Azimuth  140 
W**pen  LCPK  A***t  vXUVS 


InMntry  EIOOO 

R«ng*  3070m*l*r*  AzimuOi  130d*gr**a 
Weapon  LCPK  Aaaat  vXUVS 


Low  Inlantry  E1005 

Rang#  4072  m*4*t%  Azimuth  lAOdagi*** 
Weapon  Nona  Aaaat  Non* 


Amor  ElOOe 

Rang*  3678  maw ta  AzimuOi  141  dagwa* 
Weapon  10S  A**«t 
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Target  Engagement  Sequence 
(no  CABS) 

(7)  Once  munition  has  been  fired,  release 
Palm  Grips 

^  If  target  has  been  destroyed,  delete 
target  from  Target  Queue 

-  For  LOS  targets,  gunner  will  observe  target 
destruction 

-  For  BLOS  targets.  Observer  will  provide 
feedback  to  gunner 


Note;  When  Target  ts  removed  from  Target  Queue, 
target  is  removed  from  SA  screen 


Service  next  target 


Amor  E1003 

Rang*  4238  motor*  Amnutft  141  dogro* 
Woapon  CMS  Aooot  vXUVS 


Amor  E10O1 

Rang#  4577  motor*  AximuOr  141  dtgroo* 
Woopon  CMS  Aa*ot  vXUVS 


Amor  E1002 

Rang*  4427  motor*  AximuOt  144dogr*t 
Woopon  CMSAsoot  vXUVS 


Inlantry  E1009 

Rang*  3970 motor*  Anmugt  139d*gr*** 
Woopon  LCPK  Atoot  vXUV8 


Low  intanlry  E1005 

Rongo  4872  motor*  AiMnuSi  146  dog**** 
Woopon  NonoAaoot  Non* 


Amor  ElOOe 

Rang*  3878  motors  Aximudt  141 
Woapon  106  A«o*(  vCAVO 


Mod  Inlantry  £1007 

Rar>g*  3809  motor*  Aximud)  140  dogroo* 
Woapon  LCPK  A»»ot  vXUVS 


Fire  Permit  -  Common  States 


Not  Armed 

The  gunner  has  not  depressed  the  palm  grips  to  arm  the  system.  The  system  will  not  aim  if  it  is  not 
armed,  (^ess  frequently,  this  display  indicates  that  no  weapon  is  selected. 


Out  of  Munitions 

The  current  weapon  is  out  of  munitions,  (.ess  frequently,  the  system  is  not  armed. 


L  w  Out  of  Range 

Only  applicable  to  missile  payloads.  The  target  is  cither  too  close  or  too  far  away.  The  weapon  will  point 
in  the  direction  of  the  target  in  anticipation  of  the  target  nwving  into  range. 


Fire  Permit  Granted 

lAI 


Designator  Unavailable 

(lie  onboard  designator  can  only  fail  to  be  available  for  two  reasons:  the  target  is  beyond  its  range  or  the 
designator  is  currently  illuminating  a  different  target. 
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Target  Engagement  Sequence 
(CABS) 

O  Highlight  first  target  in  queue 
0  Select  Asset  /  Weapon 

NOTE:  Operator  may  choose  to  override 
rceommcadatioR  with  selection 

0  Select  Target  &  zoom  sensor  (if  nec) 

0  ARM  Weapon  System  by  depressing  palm 
grips  on  yoke 

NOTE:  This  will  AIM  the  Weapon  System 

0  Wait  for  Fire  Permit* 

Fire  Weapon  by  pulling  trigger 


High  Amtor  E1006 

Rang*  SOrdmalers  AzimuOt  141  d*gt**« 
W**pon  106  Aaa«  vCAVO 


ra 


Amor  E1003 

Rang*  4238  m«4*ra  Azimutft  141  d*gr*« 
Weapon  CMS  AoMt  vXUVS 


Amor  E10O1 

Rang*  4577  m*4*ra  AwngOi  141  6*9>m 
Weapon  CMS  Aaaat  vXUVS 


Amor  E1002 

Rang*  4427  malara  Azimuth  144g*gr*« 
Weapon  CMSAsaat  vXUVS 


inlantry  E10O7 

Rang*  3800m«4*t«  AzimuOt  lAOdegr*** 
Weapon  LCPK  Aeeot  vXUVS 


[  MW*  I 
I  Down  I 


Inlantry  E1009 

Rang*  9e70m*l*ra  AzimuOt  190d*gr*« 
Weapon  LCPK  Aeeet  vXUVS 


[“J 


iManlry  E10O5 

Rang*  4872  meter*  AzimuOi  14ed*gr*« 
Weapon  Non*  Aeeet  Non* 


Target  Engagement  Sequence 
(CABS) 

0  Once  munition  has  been  fired,  release 
Palm  Grips 

0  If  target  has  been  destroyed,  delete 
target  from  Target  Queue 

-  For  LOS  targets,  gunner  will  observe  target 
destruction 

-  For  BLOS  targets,  Observer  will  provide 
feedback  to  gunner 


Note;  Wtien  Target  e  removed  from  Target  Queue, 
target  is  removed  from  SA  screen 


0  Service  next  target 


Amor  ElOOe 

Rang*  3878  moat*  Azimuffi  141 
W*epon  106  Aaeat  «<rAV9 


Amor  E10O3 

Rang*  4228  m*«*(«  AzimuOi  141  d*gr*« 
Weapon  CMS  AM*t  vXUVS 


Amor  E1001 

Rang*  4577  meier*  AzimuSi  141  degieee 
Weapon  CMS  Aaeat  vXUVS 


Amor  E100S 

Rang*  4427  mewt*  Azimuth  144  degree 
Weapon  CMS  Aseat  vXUVS 


IMenlry  EIOOO 

Rang#  3970mal*r*  Azimudt  ISSdegreae 
Weapon  LCPK  Aeeet  vXUVS 


Low  Inlarary  E1005 

Rang*  4872  miMt*  Azimuth  146  degieee 
Weapon  None  Aeeet  Non* 


Mad  Inlantry  E1007 

Rang*  3808  meter*  Azimuth  140  degrees 
Weapon  LCPK  Aeeet  vXUVS 
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The  Basics 

•  CAT  (vLAV9)  and  ARV-A  (vxuvs)  are  equipped  with  weapon  payloads  and  TA  sensors 

•  ARV-R  plays  a  recon  support  role  and  may  acquire  targets 

•  Higher  echelon  COP  shared  with  CAT  crew 

•  CAT  (vLAV9) 

-  TA  sensor;  range  =  4km 

-  120mm  cannon;  18  rounds 

-  Coaxial  Machine  Gun;  2400  rounds  /  cont  fire 

•  ARV-A  (vXUV8) 

-  TA  sensor;  range  =  4km 

-  LCPK;  19  rounds,  min  range  0.5km 

-  CMS;  4  rounds,  range  1km  to  16km 


The  Basics 

•  Mission 

-  You  are  a  gunner  operating  as  part  of  3  man  MCS  crew.  You  have  been  Task  Organized 
with  an  ARV-A  and  ARV-R.  And,  you  are  providing  fire  support  for  an  RS&V  Company. 
Your  PLT  LDR  has  detected  a  potential  threat  at  approximately  2.5km  from  your  vehicle 
that  requires  additional  recon  /  surveillance.  He  has  instructed  the  ARV-R  to  investigate. 

-  Your  mission  begins  once  the  ARV-R  recon  is  complete.  Your  mission  is  to  analyze  all 
targets  that  are  available  to  you  via  the  Target  Queue  and  engage  and  defeat  as  many  of 
those  threats  as  possible,  as  quickly  as  possible. 

-  You  will  be  conducting  two  experiment  runs.  The  first  run  will  not  prioritize  any  of  the 
targets,  nor  will  it  provide  any  weapon  recommendations.  This  is  the  no  CABs  condition. 
The  second  run  will  prioritize  targets  and  provide  weapon  recommendations.  This  will  be 
the  CABs  condition.  The  overall  improvements  in  crew  performance  with  the  CABs  will  be 
measured. 
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What  Can  You  Accomplish  with  the  Lethality  Displays? 

•  Acquire  Targets 

-  Both  CAT  and  ARV-A  have  TA  sensors 

•  Engage  Targets 

-  Both  CAT  and  ARV-A  have  Direct  Fire  capability 

-  ARV-A  has  Indirect  Fire  (BLOS)  capability 


When  Would  You  Use  Lethality  Displays? 

•  Maintain  local  security  /  situational  awareness 

•  When  commanded  to  carry  out  Fire  Mission  Request 


Fight  Suite  -  Lethality  Displays 


Reports 


Target  Aequisition 


Planning 


Assets 


Target  Queue 


Planning 
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How  do  Lethality  displays  fit  in  with  the  Other  Displays? 

•  Target  Acquisition 

-  Use  to  locate,  acquire,  and  engage  targets 

•  Target  Queue 

-  Sorted  list  provides  target  type,  range,  and  firing  recommendations 

-  Select  target  in  Target  Queue  to  initiate  firing  sequence. 

•  Planning  /  SA 

-  Provides  SA  that  includes  target  location  and  proximity  to  friendlies 

•  Reports 

-  RX  Fire  Mission  Request  from  higher  echelon 

-  TX  Call  For  Effects 

•  Assets 

-  Provides  list  that  includes  assets  under  operator  control 


Fight  Suite  -  Lethality  Displays 


Reports 


Target  Aequisition 


Planning 


Assets 


Target  Queue 


Planning 
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Target  Acquisition  Display 


O  TA  Sensor  Info 

-  Asset 

-  Camera  type 

-  Zoom  level 


O  Targeting  Info 

-  Weapwn  System  status 

-  Crosshairs  /  Tracking  gates 

-  Fire  Permit 

O  Vehicle  Info 

-  Orientation  Icon 

-  Ammo  count 

-  Asset  position 

-  Weapon  Az  /  El 


Target  Acquisition  Display 


•  Both  CAT  (vLAVs)  and  ARV-A  (vxuvs) 
have  TA  Sensors.  To  utilize  a 
specific  TA  sensor,  you  must: 

0  Select  Asset 

0  Take  Control  of  Weapon  System 

(Weapon  System  stale  transitions  from  SAFE  to  READY) 

NOTE:  You  must  have  acquired  an 
Asset  to  be  able  to  take  control  of  the 
weapon  system 


0  Select  Weapon 
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Intentionally  left  blank 
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Appendix  B.  Surveys 


Participant  ID; 


General: 


Date: _  Armband: 

Demographic  Questionnaire 


I.  DOB; _  2.  Sex:  M  /  F  3.  Height: _ ft.  _ in. 

4.  Weight: _ lbs.  5.  Handedness:  Right /Left  6.  Smoker;  Y/N 

7.  Rank: _ 

8.  Time  in  service:  Yrs: _ Mos: _ 

9.  Time  in  grade:  Yrs: _ Mos: _ 

10.  MOS/Speeialty: _ 

II.  Time  in  MOS/Speeialty:  Yrs: _ Mos: _ 

12.  Combat  experienee:  Y/N  If  Yes,  Where? _  How  long? _ 


Experimentation: 

9.  How  many  UAMBL  experiments  (i.e.,  simulations  in  the  battle  lab)  have  you  partieipated  in? 

(if  none,  indieate  0):  _ 

10.  Indieate  whether  you  have  had  experienee,  and  at  what  level,  for  eaeh  the  following: 

a.  Use  of  live  CAT-Styrker  vehieles 

_ None _ Basie  _ Intermediate _ Advaneed 

b.  Use  of  simulated  CAT-Styrker  vehieles 

_ None _ Basie  _ Intermediate _ Advaneed 

e.  Control  of  live  unmanned  systems  (e.g.,  UAVs,  UGVs,  other) 

_ None _ Basic  _ Intermediate _ Advaneed 

d.  Control  of  simulated  unmanned  systems  (e.g.,  UAVs,  UGVs,  other) 

_ None _ Basie  _ Intermediate _ Advaneed 

e.  Use  of  MC2  (Maneuver  Command  and  Control) 

_ None _ Basie  _ Intermediate _ Advaneed 

f.  Use  of  FBCB2  (Foree  XXI  Battle  Command  Brigade  and  Below) 

None  Basie  Intermediate  Advaneed 
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g.  Use  of  COP  (common  operational  picture  interface)  in  CAT  Stryker  (live  or  simulated) 
_ None _ Basic  _ Intermediate _ Advanced 

ti.  Do  you  have  any  experience  with  computer  games  where  you  control  a  vehicle? 

None  Basic  Intermediate  Advanced 


NASA  TLX  Workload  Assessment  Instructions 

We  are  interested  in  the  “workload”  you  experienced  during  this  scenario.  Workload  is 
something  experienced  individually  by  each  person.  One  way  to  find  out  about  workload  is  to 
ask  people  to  describe  what  they  experienced.  Workload  may  be  caused  by  many  different 
factors  and  we  would  like  you  to  evaluate  them  individually.  The  set  of  six  workload  rating 
factors  was  developed  for  you  to  use  in  evaluating  your  experiences  during  different  tasks. 
Please  read  them.  If  you  have  a  question  about  any  of  the  scales  in  the  table,  please  ask  about  it. 
It  is  extremely  important  that  they  be  clear  to  you. 


Definitions 


Title 

End  points 

Descriptions 

MENTAL 

DEMAND 

Low  /  High 

How  much  mental  and  perceptual  activity  was  required  (that  is,  thinking, 
deciding,  calculating,  remembering,  looking,  searching,  etc.)?  Was  the  task 
easy  or  demanding,  simple  or  complex,  exacting  or  forgiving? 

PHYSICAL 

DEMAND 

Low  /  High 

How  much  physical  activity  was  required  (that  is,  pushing,  pulling,  turning, 
controlling,  activating,  etc.)?  Was  the  task  easy  or  demanding,  slow  or  brisk, 
slack  or  strenuous,  restful  or  laborious? 

TEMPORAL 

DEMAND 

Low  /  High 

How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace  at  which  the 
tasks  or  task  elements  occurred?  Was  the  pace  slow  and  leisurely  or  rapid 
and  frantic? 

PERFORMANCE 

Poor / Good 

How  successful  do  you  think  you  were  in  accomplishing  the  goals  of  the 
task?  How  satisfied  were  you  with  your  performance  in  accomplishing  these 
goals? 

EFFORT 

Low  /  High 

How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  your 
level  of  performance? 

FRUSTRATION 

LEVEL 

Low  /  High 

How  insecure,  discouraged,  irritated,  stressed,  and  annoyed  versus  secure, 
gratified,  content,  relaxed  and  complacent  did  you  feel  during  the  task? 

We  want  you  to  evaluate  workload  in  two  different  ways.  First,  rate  the  workload  on  each  factor 
on  a  scale.  Each  scale  has  two  end  descriptions,  and  20  slots  (hash  marks)  between  the  end 
descriptions.  Place  an  “x”  in  the  slot  (between  the  hash  marks)  that  you  feel  most  accurately 
reflects  your  workload. 

Next,  we  want  you  to  compare  the  various  workload  factors.  This  comparison  is  a  technique 
developed  by  NASA  to  evaluate  the  relative  importance  of  the  six  workload  sources  you  used  to 
rate  the  workload  you  experienced.  The  procedure  is  simple:  you  have  a  sheet  with  a  series  of 


36 


pairs  of  workload  sources  (for  example,  Effort  vs.  Mental  Demands).  We  want  you  to  ehoose 
whieh  of  the  sourees  was  more  important  to  your  experienee  of  the  workload  in  the  task  that  you 
performed.  You  will  fill  out  a  separate  sheet  for  eaeh  task. 

For  eaeh  pair  of  sourees,  eirele  the  souree  that  is  the  more  important  eontributor  to  the  workload 
for  the  speeifie  task  you  are  rating.  For  example,  for  the  first  eomparison,  if  effort  eontributes  to 
workload  more  than  performanee,  eirele  effort. 

After  you  have  finished  the  entire  series,  we  will  be  able  to  use  the  pattern  of  your  ehoiees  to 
ereate  a  weighted  eombination  of  ratings  into  a  summary  workload  seore. 

We  ask  you  to  evaluate  your  workload  for  this  seenario.  This  ineludes  all  the  duties  involved  in 
your  job  (e.g.,  preparing  your  workstation,  using  displays  and  eontrols  at  your  workstation). 
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Participant  ID: _  Date: _  Time: _  Experiment: 

Condition: 


TLX  Workload  Scale 

Please  rate  your  workload  by  putting  a  mark  on  eaeh  of  the  six  seales  at  the  point  whieh  matehes 
your  experienee. 


Mental  Demand 


Very  Low  Very  High 


Physical  Demand  — ' — — ^ ^ ^ — — ' — — ^ ^ ^ — — ' — — ^ — 

Very  Low  Very  High 


Temporal  Demand  — ' — — I — — I — — I — — I — — I — — I — — I — — I — — I — 

Very  Low  Very  High 


Performance 


Very  Poor 


Very  Good 


Effort  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I 

Very  Low  Very  High 


Frustration 


Very  Low 


Very  High 
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Participant  ID; _  Date: _  Time: _  Experiment: 

Condition: 


Subjective  Stress  Rating  Scale 

1.  The  seale  below  represents  a  range  of  how  PHYSICALLY  stressful  the  mission  might  be. 
Check  the  bloek  indieating  how  PHYSICALLY  stressful  the  mission  you  just  partieipated  in 
was. 


Task 

Not  at 
All 

Stressful 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Most 

Possible 

Stress 

10 

a.  Overall  stress 

2.  The  seale  below  represents  a  range  of  how  MENTALLY  stressful  the  mission  might  be. 
Cheek  the  bloek  indieating  how  MENTALLY  stressful  the  mission  that  you  just  partieipated  in 
was. 


Task 

Not  at 
All 

Stressful 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Most 

Possible 

Stress 

10 

a.  Overall  stress 
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Participant  ID; _  Date: _  Time: _  Experiment: 

Condition: 


Situation  Awareness  Rating  Scale 


► 


► 


No 


My  SA  with  respect  to  the  task  was  excellent. 

I  was  able  to  perform  the  task  extremely  well 
all  of  the  time. 

10 

My  SA  with  respect  to  the  task  was  very  good. 

I  was  able  to  perform  the  task  well  all  of  the  time. 

9 

My  SA  with  respect  to  the  task  was  good. 

I  was  able  to  perform  the  task  well  most  of  the  time. 

8 

My  SA  with  respect  to  the  task  was  not  complete. 

I  was  able  to  perform  the  task,  but  not  satisfactorily. 

7 

My  SA  with  respect  to  the  task  was  insufficient. 

I  was  not  aware  of  all  the  information  required  to 
perform  the  task  effectively. 

6 

My  SA  with  respect  to  the  task  was  reduced. 

I  was  unaware  of  some  of  the  important 
information  required  to  perform  the  task  effectively. 

5 

My  SA  with  respect  to  the  task  was  low. 

I  was  unaware  of  about  half  of  the  information 
required  to  perform  the  task  effectively. 

4 

My  SA  with  respect  to  the  task  was  very  low. 

I  was  unaware  of  most  of  the  information  required 
to  perform  the  task  effectively. 

3 

My  SA  with  respect  to  the  task  was  extremely  low. 

I  was  unaware  of  almost  all  of  the  information 
required  to  perform  the  task  effectively. 

2 

My  SA  with  respect  to  the  task  was  far  too  low. 
I  could  not  perform  the  task  because  I  did  not 
possess  the  necessary  information. _ 
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1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
ONLY)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

I  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BELVOIR  VA  22060-5608 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARE  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

2  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARE  CS  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  M  DR  M  STRUB 
6359  WALKER  LANE  SUITE  100 
ALEXANDRIA  VA  22310 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  ML  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MC  A  DAVISON 
320  MANSCEN  LOOP  STE  166 
FT  LEONARD  WOOD  MO  65473-8929 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 


I  COMMANDANT  USAADASCH 
ATTN  ATSA  CD 

ATTN  AMSRD  ARE  HR  ME  MS  A  MARES 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MI  J  MINNINGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MM  DR  V  RICE 
BLDG  4011  RM2I7 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5094 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

10  ARE  HRED  ARMC  FED  ELMT 

ATTN  AMSRD  ARE  HR  MH  B  STERLING 
BLDG  I467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40 12 1 

I  ARMY  RSCH  LABORATORY  -  HRED 
AVNC  FIELD  ELEMENT 
ATTN  AMSRD  ARE  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

I  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
9I0I2  STATION  AVE  ROOM  III 
FT  HOOD  TX  76544-5073 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1172  BLDG  51005 
FTHUACHUCAAZ  85613-7069 

I  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 
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ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  M  DR  B  KNAPP 
ARMY  GI  MANPRINT  DAPE  MR 
300  ARMY  PENTAGON  ROOM  2C489 
WASHINGTON  DC  20310-0300 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MJK  MS  D  BARNETTE 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  SS  E  BLDG  3  RM  341 
NATICK  MA  01760-5020 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MT  DR  J  CHEN 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826-3276 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  RM  303A 
FORT  GORDON  GA  30905-5233 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MU  M  SINGAPORE 
6501  E  II  MILE  RD  MAIL  STOP  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MF  MR  C  HERNANDEZ 
BLDG  3040  RM  220 
FORT  SILL  OK  73503-5600 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FTBENNING  GA  31905-5400 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 


I  NIGHT  VISION  DIRECTORATE 
DEPARTMENT  OF  ARMY 
ATTN  AMSRD  CR  NV  CM  DSA 
A  L  SANTIAGO 
1 022 1  BURBECK  ROAD 
FORT  BELVOIR  VA  22060-5806 

I  SYSTEMS  INTEGRATION  DIVISION 
US  ARMY  ENGINEER  SCHOOL 
ATTNATSEDE  J  BEACHAM 
320  MANSCEN  LOOP  SUITE  235 
FORT  LEONARD  WOOD  MO  65473 

ABERDEEN  PROVING  GROUND 

I  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARE  Cl  OK  TECH  LIB 
BLDG  4600 

I  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARE  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

I  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARE  HR  MR  F  PARAGALLO 

BLDG  459 
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